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FOREWORD 


This  report  was  written  at  the  request  of  the  Commanding  Officer  and  Technical 
Director  of  Navy  Personnel  Research  and  Development  Center  to  review  the  current  state 
of  computer-based  instruction.  It  briefly  summarizes  the  effectiveness  of  current 
computer-based  instruction  programs,  where  these  programs  are  being  used  and  by  whom, 
and  recommends  how  to  incorporate  the  technology  into  Navy  training.  The  results  are 
intended  for  a  general  audience  concerned  with  the  use  of  computer-based  training 
systems  in  military  training. 


:.  W.  RENARD 
Captain,  U.S.  Navy 
Commanding  Officer 


1.  W.  TWEEDDALE 
Technical  Director 


SUMMARY 


Purpose  and  Background 

This  report  briefly  summarizes  the  current  state  of  computer-based  instruction  (CBI) 
programs,  discusses  where  these  programs  are  being  used  and  by  whom,  and  recommends 
how  to  incorporate  the  technology  into  Navy  training. 

Often  thought  of  as  a  quick  technical  solution  to  Navy  training  problems,  the 
computer  can  be  programmed  to  be  a  substitute  teacher  and  to  represent  a  task. 
However,  the  difficulties  in  describing  and  designing  teaching  interactions  and  program¬ 
ming  computers  to  control  the  problems  are  enormous.  For  20  years,  computer-based 
systems  have  been  tested  that  resemble  automated  and  animated  textbooks  with  on-line 
assessment  of  learning;  these  systems  can  be  implemented  now. 

Recent  Research  on  Computer-Based  Instruction  (CBI) 

In  more  than  100  comparisons  of  CBI  with  traditional  (lecture)  instruction,  re¬ 
searchers  found  that  the  achievement  of  students  using  CBI  were  as  good  and  often  better 
than  the  achievements  of  students  taught  with  traditional  classroom  methods.  Also, 
comparisons  showed  that  students  learned  more  than  30  percent  faster  using  CBI  than 
with  traditional  methods. 

Many  firms  have  adopted  CBI  for  training.  These  firms  range  from  airlines  to 
insurance  companies,  banks,  and  federal  agencies.  The  increasing  use  of  CBI  by  industrial 
organizations  indicates  confidence  in  its  usefulness  for  training.  These  firms  invest 
capital  resources  based  on  analysis  of  how  CBI  meets  their  training  requirements. 
Educational  systems  are  also  investing  public  funds  to  improve  the  effectiveness  and 
efficiency  of  teaching. 

Results  and  Discussion 


Several  problems  must  be  solved  before  CBI  can  be  used  for  technical  training  in  the 
Navy.  First,  the  characteristics  of  the  CBI  programs  must  be  matched  to  the  training 
requirements,  which  require  an  effective  planning  system.  Second,  an  organization  .must 
collect  and  collate  effective  existing  CBI  programs  and  provide  standardization  and 
distribution.  As  new  programs  are  developed,  a  system  is  needed  that  aids  and  facilitates 
the  instructional  program  development  process.  This  system  should  be  computer-based  to 
increase  its  use  and  to  control  the  process. 

Current  textbook-like  CBI  offers  limited  gains  that  result  mainly  from  testing  and 
better  management  of  students'  time.  However,  an  insufficient  base  of  scientific 
knowledge  hampers  the  development  and  usefulness  of  advanced  CBI  systems.  A 
systematic  research  program  is  needed  to  improve  this  knowledge  base  and  to  improve 
CBI  technology. 

Conclusions 


CBI  in  its  current  form  can  be  useful  for  certain  Navy  technical  training,  for 
example,  at  remote  sites  where  instructors  are  unavailable.  It  can  be  used  to  improve 
training  or  ensure  training  is  successful  and  of  high  quality.  The  technology  requires 
systematic  development,  and  its  use  requires  careful  analysis  of  the  requirements  for 
training  supported  by  CBI. 


Recommendations 


1.  Develop  usable  procedures  to  determine  when  and  how  to  employ  CBI  effectively 
for  teaching  courses  and  to  manage  its  widespread  distribution. 

2.  Support  the  development  of  automated  systems  to  improve  instructional  program 
development  and  systems  for  collecting  and  distributing  effective  programs. 

3.  Support  programmatic  research  efforts  to  uncover  and  apply  new  knowledge  that 
will  make  CBI  increasingly  effective  for  teaching  complicated  tasks. 
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INTRODUCTION 


Purpose  and  Background 

This  report  briefly  summarizes  current  computer-based  instruction  (CBI)  programs, 
their  effectiveness,  and  uses. 

The  potential  ability  of  CBI  to  manage  teaching  has  been  apparent  for  many  years. 
The  difficulty  in  programming  a  system  to  provide  the  experiences  necessary  for  effective 
learning,  either  as  a  supplement  to  or  as  a  substitute  for  a  master  teacher,  has  not  been 
apparent.  The  increasing  availability  of  relatively  inexpensive  microcomputer  power 
makes  this  potential  attractive  because  it  seems  affordable.  People  fail  to  consider  that 
computers  alone  do  not  contain  all  the  ingredients  needed  for  successful  instruction. 
Those  ingredients  are  provided  currently  by  crude  instructional  technology  that  often 
provides  insufficient  detail  of  what  needs  to  be  taught,  from  which  prescriptions  are 
made,  and  how  to  structure  and  sequence  the  interactions  with  students  to  promote 
learning.  Then,  for  CBI,  the  specifications,  prescriptions,  and  materials  must  be 
programmed  so  a  computer  can  control  the  process.  Computer  capabilities  have  improved 
to  the  point  that  one  might  imagine  could  be  used  to  provide  a  sophisticated  learning 
environment  rivaling  that  of  real  interactions  with  tutors  and  tasks.  Unfortunately, 
before  the  requirements  for  such  programs  can  be  specified,  considerable  research  is 
required  that  explores  the  relation  between  students'  mental  operations,  achievement,  and 
instructional  methods  in  learning  complicated  tasks. 

Computers  do  not  provide  guidance  for  large-scale  implementation  of  CBI  into 
instructional  programs.  Their  use  poses  substantial  planning  problems,  including  the  need 
for  space,  power,  instructor  training  and  assignment,  student  familiarization,  etc. 
Considerable  time  and  energy  are  needed  to  plan  programs  properly.  Various  reasons  for 
using  computers  in  training  situations,  other  than  those  in  direct  instructional 
interactions,  are  shown  in  the  following  list. 

1.  Administrative  purposes.  Accounting,  payrolls,  employee  records,  student 
recordkeeping,  attendance,  grades,  timetables,  and  planning  systems. 

2.  Curriculum  planning.  Resource  information  keeping  and  planning,  production  of 
instructional  materials,  and  test  construction. 

3.  Professional  (staff)  development  and  training. 

4.  Library  and  data  base  search  for  information. 

5.  Evaluation  of  programs. 

The  instructional  uses  of  computers  can  vary  as  well.  Most  CBI  has  been  in  the  form 
of  an  automated  workbook,  presenting  text  to  be  read  and  questions  to  test  comprehension 
of  the  text.  When  appropriate,  pictures  and  graphs  are  included.  Recently,  videotapes 
and  videodiscs  have  been  added  to  provide  dynamic  demonstrations  and  have  improved 
interactivity.  Computer  graphics  provide  other  ways  to  represent  visual  information,  but 
require  considerable  expertise  in  computer  programming  and  how  best  to  represent  such 
information  to  students.  Voice  synthesizers  make  audio  presentations  possible. 

Use  of  computers  in  training  requires  talents  to  analyze  and  define  requirements, 
determine  how  instruction  should  occur,  match  requirements  with  capabilities  of 
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computer  systems,  and  program  them.  Therefore,  use  of  computers  in  military  training 
must  be  guided  by  cautious  design  plus  planning  to  introduce  CBI  gradually  into 
instructional  programs.  Such  talents  are  not  readily  available.  Support  must  be  provided 
for  programmatic  research  and  development  efforts  to  evolve  the  instructional  and 
software  technologies  to  guide  the  use  of  CBI  and  to  enhance  the  effectiveness  of  the 
available  talent. 


RECENT  RESEARCH  ON  COMPUTER-BASED  INSTRUCTION  (CBI) 

Effectiveness 


In  general,  well  designed  and  appropriately  managed  CBI  does  instruct  as  effectively 
as  well  designed  traditional  methods.  CBI  is  more  efficient  than  alternative  ways  of 
delivering  the  same  content.  In  a  few  exemplary  cases,  CBI  provides  instruction 
impossible  with  other  means.  Whether  CBI  is  worth  the  cost  is  an  analytical  problem  that 
needs  more  attention  (Orlansky  5c  String,  1980). 

During  the  last  15  years,  many  evaluations  have  been  made  of  the  effectiveness  and 
efficiency  of  CBI  relative  to  standard,  fixed-time  lecture  courses.  Most  of  these 
evaluations  have  compared  computerized  versions  of  programmed  instruction  with  content 
delivered  by  lecture  and  text.  The  usual  variables  measured  in  the  comparisons  were 
student  achievement  on  accepted  achievement  tests,  how  long  students  took  to  complete 
the  materials,  and  whether  students  or  instructors  liked  it.  Some  gain  in  effectiveness 
was  found,  for  example,  achievement  test  scores  for  CBI  students  were  the  same  or  a 
little  better,  but  the  overwhelming  finding  was  that  students  completed  the  materials 
faster.  Students  tended  to  like  CBI  because  it  involved  them  in  the  learning  process. 
Instructors,  on  the  other  hand,  were  ambivalent  and  often  negative  because  they  perceive 
CBI  as  usurping  their  role  or  complicating  their  lives.  The  following  items  summarize 
some  evidence  specifically  concerned  with  the  effectiveness  of  CBI,  for  example,  whether 
it  teaches  as  well  as  usual  methods. 

•  In  comparisons  done  in  48  military  training  courses,  achievement  was  about  the 
same  in  32  cases,  slightly  superior  in  15  cases,  and  slightly  poorer  in  1  case.  In  44  cases, 
completion  time  was  30  percent  faster  for  the  computer-based  courses  than  for  the 
standard  courses  (Orlansky  &  String,  1980). 

•  Careful  implementation  of  CBI  in  military  courses  has  had  positive  results  on 
factors  related  to  effective  teaching.  In  contrast  to  a  traditional  course,  CBI  provided 
better  management  of  the  students  and  increased  their  time  studying  the  material,  their 
contact  with  equipment  they  must  learn  to  use,  and  their  interactions  with  instructors  on 
the  course  topics  (Van  Kekerix,  Wulfeck,  <5c  Montague,  1982).  Using  CBI,  instructors  could 
use  their  time  normally  spent  on  nonproductive  activities,  like  scoring  tests,  for  more 
productive  teaching  tasks.  If  this  extra  time  was  used  by  teachers  to  tutor  students, 
students  would  benefit  substantially.  Another  recent  study  showed  that  students  learned 
quickly  to  recognize  radar  jamming  by  using  CBI  (McDonald  &  Crawford,  1983).  In  this 
evaluation,  CBI  provided  the  opportunity  to  practice  rarely  encountered  events  (in 
peacetime)  and  an  opportunity  to  use  teaching  techniques  not  available  with  actual 
equipment.  However,  management  must  carefully  consider  the  use  of  CBI  systems  to 
enhance  teaching;  the  Navy  has  not  been  entirely  successful  in  enhancing  their  instruc¬ 
tional  program  development  (Montague  &  Wulfeck,  1982). 
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•  Greater  effectiveness  of  CBI  was  found  in  an  analysis  of  59  independent  evalua¬ 
tions  of  CBI  in  college  courses  (Kulik,  Kulik,  &  Cohen,  19S0)  and  in  an  analysis  of  51 
independent  evaluations  of  teaching  in  grades  6  to  12  (Kulik,  Bangert,  <5c  Williams,  1983). 
For  the  college  courses,  CBI  significantly  improved  students'  achievement  test  perfor¬ 
mance  in  37  comparisons.  In  grades  6  to  12,  the  CBI  class  was  superior  in  39  of  48  studies 
measured  by  final  examination  scores.  For  a  typical  CBI  class,  scores  were  raised  about 
.25  standard  deviations  in  the  college  courses  and  .32  standard  deviations  in  grades  6  to 
12.  This  means  that  in  the  two  analyses,  the  average  student  in  CBI  classes  outperformed 
60  and  63  percent  of  the  students  in  conventional  classes.  There  was  also  a  positive  gain 
in  the  attitudes  of  students  toward  instruction  and  toward  the  subject  matter.  Only  eight 
evaluations  in  the  college  courses  and  two  in  grades  6  to  12  were  concerned  with  time 
spent  in  instruction  (academic  education  organized  around  quarters,  semesters,  or  school 
years  seems  not  to  be  concerned  about  efficiency).  All  10  studies  reported  a  time  savings 
of  33  percent  or  greater,  which  is  a  substantial  improvement  and  agrees  with  the  military 
studies  cited  earlier.  Some  CBI  classes  might  have  shown  both  performance  gains  and 
time  savings,  but  the  report  is  unclear  on  this  point. 

Better  programming  methods  developed  in  artificial  intelligence  work,  theoretical 
efforts  in  cognitive  science,  and  hardware  developments  have  stimulated  the  recent 
development  of  generic  and  intelligent  CBI  systems.  These  programs  have  the  potential 
to  approximate  the  effectiveness  of  tutorial  instruction.  Such  systems  can  compare 
student  performance  with  computer  models  of  students  and  experts,  can  infer  why  a 
student  errs  and  what  is  needed  to  promote  learning,  and  can  generate  the  next 
interaction.  The  presentations  to  the  student  are  not  rigidly  preprogrammed.  These 
approaches  typically  will  not  encompass  an  entire  course,  but  will  focus  on  critical  task 
training.  Thus,  they  can  be  included  in  any  system  of  course  management!  A- large 
research  and  development  effort  is  needed  to  develop  this  technology. 

•  A  potentially  effective  use  of  CBI  is  as  a  substitute  for  costlier  (and  often 
unavailable)  real  equipment  or  special  simulators.  Airline  and  military  pilots  are  using 
CBI  in  addition  to  flight  simulators,  and  substantial  savings  are  claimed  for  this  use  of 
CBI.  One  of  the  best  current  examples  of  a  simulated  training  system  is  STEAMER,  which 
trains  operators  of  1200-pounds-per-square-inch  steam  power  plants  for  ships  (Hollan, 
Stevens,  &  Williams,  1980;  Hollan,  Hutchins,  &  Weitzman,  1984).  Although  its  instruc¬ 
tional  effectiveness  has  not  been  evaluated,  STEAMER  has  the  potential  to  provide 
training  not  possible  with  current  resources  that  will  substantially  improve  performance. 
Another  example  is  the  present  use  of  a  microcomputer  to  successfully  train  radar 
operators  to  recognize  types  of  jamming  (McDonald  &  Crawford,  1983). 

•  Another  effective  use  of  CBI  is  the  maneuvering  board  trainer  (Hutchins  & 
McCandless,  1982).  This  system  trains  Navy  officers  and  enlisted  personnel  on  a  task  that 
has  historically  been  difficult  to  learn.  Using  simulation  techniques  derived  from 
cognitive  psychology  and  artificial  intelligence,  the  system  provides  extensive  practice 
and  explanation  of  difficult  navigation  problems.  Initial  tests  show  that  failing  students 
can  be  brought  up  to  passing  grades  in  less  than  a  third  of  the  time  normally  taken  in  the 
traditional  course. 

Users 


The  increasing  use  of  CBI  by  schools  and  industry  for  training  and  education  is  a 
strong  indication  of  its  success  and  usefulness.  Selected  examples  of  these  uses  of  CBI 
are  discussed  in  three  groups:  (1)  profit-making  organizations  that  risk  capital  to  gain 
efficiencies  and  provide  distribution  of  training,  (2)  educational  organizations  that  invest 
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public  funds  to  improve  effectiveness,  and  (3)  sellers  and  distributors  of  training 
programs.  For  a  more  complete  list  of  users  and  distributors,  see  Kearsley  (1983). 

Profit-Making 

•  IBM  uses  CBI  to  train  about  10,000  field  engineers  around  the  country  on  more  than 
400  remote  terminals.  These  are  located  in  IBM  offices  and  provide  instruction  on  the 
function  and  maintenance  of  equipment.  Materials  are  in  text  form  (manuals),  while 
testing  is  on-line  to  assess  mastery  of  materials.  IBM  indicates  significant  savings  in 
travel  and  per  diem  costs. 

•  Insurance  companies  and  banks  use  CBI  to  train  claim  processing  agents  and  tellers. 
One  goal  is  that  training  be  the  same  at  all  the  locations.  In  1978,  Western  Bank 
Corporation  installed  a  financial  terminal  system  and  developed  a  CBI  course  to  put  on 
the  new  terminals  (Rahmelow,  1979).  This  course  trained  6800  tellers  and  1500  bank 
officers  at  various  locations  to  use  the  new  system. 

•  American  Airlines  estimates  that  CBI  and  simulation  reduced  their  training  time  by 
50  percent  and  saved  $30  million  in  fuel  (Kearsley,  1983).  The  Navy  has  justified  its  use 
of  computer-based  training  devices  for  aircrews  on  the  same  basis. 

•  United  Airlines  plans  to  do  all  its  flight  training  for  the  Boeing  767  on  a  CBI  system 
with  simulation,  thereby  reducing  the  use  and  cost  of  special  trainers  and  flying  time. 
Airlines  also  use  terminal  systems  to  train  (and  upgrade)  people  to  book  reservations, 
calculate  fares,  and  learn  other  skills. 

•  Holiday  Inns  train  management  personnel  in  operations  and  upgrade  their  training 
as  required  on  a  network  of  terminals.  The  system  is  designed  to  teach  management 
planning  skills. 

•  The  Bell  System  uses  CBI  widely  for  training  personnel  to  install  and  maintain  new 
equipment.  They  point  to  savings  in  time  and  travel. 

•  The  oil  and  gas  industry,  similarly,  provides  training  at  remote  locations  (training 
on  demand)  for  their  personnel  (Rebstock,  1980).  This,  too,  saves  travel  costs. 

•  Honeywell  uses  CBI  to  train  people  to  use  terminals,  time-sharing  systems, 
operating  systems,  programming,  report  generation,  etc.  The  Digital  Equipment  Corpora¬ 
tion  uses  CBI  to  train  personnel  after  purchasing  new  systems. 

•  Corporations  have  been  using  CBI  programs  to  upgrade  the  training  of  engineering 
technicians  because  of  rapid  technology  changes  (Modesitt,  1981). 

•  Many  departments  of  the  Federal  government  use  CBI  for  initial  training  and 
upgrading  of  their  personnel  (e.g.,  Federal  Aviation  Administration,  Internal  Revenue 
Service,  Department  of  Education,  Customs  Service,  Social  Security  Administration,  and 
Federal  Housing  Administration).  These  agencies  adopted  CBI  because  of  studies  showing 
that  CBI  is  cost-effective  for  their  application  compared  to  conventional  training 
(Orlansky,  personal  communication).1 


13esse  Orlansky,  Institute  for  Defense  Analyses,  Arlington,  Virginia,  phone  conversa¬ 
tion,  1984. 
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Educational 


•  School  systems  are  purchasing  networked  CBI  for  mathematics  and  basic  literacy 
training.  Over  150,000  students  receive  daily  training  in  24  states,  and  this  has  led  to 
significant  gains  on  achievement  tests  (Suppes,  1979). 

•  The  Philadelphia  School  District  is  one  of  several  districts  that  have  been  using  CBI 
in  education  for  years.  In  1969,  Philadelphia  secondary  schools  taught  biology  and  reading 
courses  with  CBI,  and,  during  the  1970s,  these  reading  programs  were  used  in  both 
elementary  and  secondary  levels.  Hundreds  of  students  use  the  CBI  programs  to  meet 
reading  comprehension  requirements.  Recently  these  programs  have  been  made  available 
on  common  microcomputers  (Kearsley,  Hunter,  &  Seidel,  1983). 

•  The  National  Development  Programme  in  Computer-Assisted  Learning  Project  in 
England  developed,  with  government  support,  35  courses  at  47  different  educational 
institutions.  A  specific  goal  was  to  produce  generalizable,  usable  programs  that  would  be 
used  after  the  project  support  stopped.  Currently,  most  of  those  courses  developed  are 
being  used  with  the  costs  shared  among  institutions.  The  materials  are  now  part  of  the 
regular  instruction,  and  the  costs  are  incorporated  into  regular  budgeting  (Fielden  & 
Pearson,  1978;  Hooper,  1977;  Kearsley  et  al.,  1983). 

Sellers  and  Distributors 

Another  indication  of  public  acceptance  is  that  major  firms  have  been  successfully 
marketing  CBI  systems  and  programs  for  over  10  years  and  have  a  multitude  of  users. 

•  PLATO  began  in  the  early  1960s  at  the  University  of  Illinois  and  developed  into  a 
large-scale  CBI  system  by  the  early  1970s.  It  was  adapted  for  commercial  use  by  the 
Control  Data  Corporation,  which  now  sells  computers  and  various  training  programs  to 
businesses,  schools,  and  individuals  through  local  learning  centers.  The  system  has  been 
transferred  to  microcomputers  for  wider  distribution. 

•  The  TICCIT  system  was  developed  as  another  large-scale  time-sharing  system  using 
color  TV  both  for  text  and  dynamic  imagery.  It  uses  a  different  instructional  approach 
than  other  systems  to  make  preparation  of  materials  easier.  It  is  currently  being 
marketed  for  microcomputers  and  minicomputers  by  Hazeltine,  Inc.  The  Navy  uses  a 
TICCIT  system  for  part  of  an  aircrew  training  course. 

•  Computer  Curriculum  Corporation  sells  programs  in  mathematics  and  literacy 
instruction  to  school  districts  around  the  country  (Suppes,  1979). 

•  The  increasing  availability  of  inexpensive  microcomputers  has  created  a  market  for 
instructional  software.  Over  50  percent  of  U.S.  schools  have  such  systems  and  do  training 
on  them.  The  problem  in  such  use  of  computers  is  keeping  track  of  the  software  and 
courseware  available  for  the  various  machines.  Various  groups  and  organizations  provide 
indexing  and  cataloging  of  instructional  programs.  For  example,  ENTELEK  CAI/CMI 
Catalog  and  the  Index  to  Computer-Based  Learning  were  early  attempts  to  provide  this 
information  (Lekan,  1968;  Wang,  1976).  Large-scale  efforts  to  distribute  programs  are 
CONDUIT,  EDUCOM,  and  MECC.  CONDUIT  began  in  the  early  1970s  at  the  University  of 
Iowa.  It  reviews  and  distributes  courseware  packages,  including  student  and  instructor 
guides  for  college  courses.  EDUCOM  was  established  in  the  1960s  to  promote  sharing  of 
computer  resources  among  members  (currently  about  350  colleges  and  universities).  This 
is  now  done  electronically  over  a  computer  network.  The  Minnesota  Educational 
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Computing  Consortium  provides  support  services  and  computer  program  distribution  to 
public  school  systems.  More  than  3000  of  their  microcomputer  systems  are  used  in  437 
school  districts,  reaching  about  95  percent  of  primary  and  secondary  students. 


RESULTS  AND  DISCUSSION 

This  brief  review  indicates  there  is  sufficient  confidence  in  the  efficiency  and 
effectiveness  of  CBI  for  profit-making  and  publicly  funded  organizations  to  invest  in 
widespread  implementation.  Research  shows  that  in  contrast  with  standard,  lecture-type 
instruction,  CBI  results  in  equal  or  somewhat  better  achievement.  Also,  students  who  use 
CBI  may  take  30  percent  less  time  to  complete  a  course  than  students  with  conventional 
instruction.  Thus,  students  can  perform  useful  work  faster,  thereby  returning  value  for 
the  cost  of  their  training.  Most  of  the  evidence  cited  compared  large  segments  or  courses 
of  instruction  taught  conventionally  or  by  CBI.  The  most  valuable  uses  of  CBI,  however, 
may  be  as  devices  for  critical  part-task  simulations  rather  than  complete  instructional 
delivery  systems.  Such  systems  provide  a  tutorial  learning  environment  for  students  and 
should  yield  substantial  gains  in  effectiveness  in  contrast  to  conventional  CBI  or 
classroom  instruction.  The  difficulty  in  analyzing  the  learning  process  and  task  necessary 
for  programming  them  makes  them  rare  and  expensive.  Therefore,  careful  planning  is 
necessary  to  develop  appropriate  CBI,  incorporate  it  into  training  programs,  develop 
procedures  for  training  managers  and  instructors,  and  maintain  the  systems. 

Implicit  or  explicit  in  the  discussion  so  far  is  that  there  are  many  reasons  for  using 
CBI,  most  of  which  are  summarized  below. 

1.  Provide  students  direct  contact  with  tasks  to  be  learned  via  simulations. 

2.  Reduce  the  variability  of  instruction. 

3.  Improve  security  and  quality  of  achievement  testing. 

4.  Reduce  requirements  for  presence  of  instructors. 

5.  Make  training  available  at  anytime  to  match  student  availability. 

6.  Provide  training  at  remote  sites  where  instructors  are  unavailable. 

7.  Reduce  training  time. 

8.  Convenience. 

9.  Reduce  travel  requirements  and  per  diem  expenses  for  training. 

10.  Reduce  development  time  for  teaching  materials  and  revisions. 

Except  for  the  first  two,  the  reasons  are  concerned  with  the  management  of  training  in  a 
general  sense  and  not  with  the  effectiveness  of  the  learning  and  instruction  process. 

The  users  of  CBI  already  mentioned  adopted  it  after  careful  study  of  their 
requirements  and  matching  them  with  the  capabilities  of  current  systems.  Similarly,  if 
existing  CBI  programs  for  microcomputers  are  to  be  incorporated  into  Navy  instructional 
programs,  the  decision  to  use  them  should  be  guided  by  the  knowledge  and  analysis  of  how 
and  why  they  were  developed,  whether  their  objectives  meet  current  needs,  and  evidence 
of  their  effectiveness  and  relative  efficiency.  Because  programs  or  courseware  purchased 
off  the  shelf  often  lack  information  about  their  purpose  and  effectiveness,  their 
introduction  may  actually  interfere  with  or  reduce  the  quality  of  instructional  programs. 
A  major  problem  with  using  traditional  CBI  is  that  prospective  users  are  unaware  of  the 
limitations  of  existing  techniques  and  programs. 

Even  if  requirements  for  instruction  are  carefully  identified  and  instruction  has  been 
well  designed,  CBI  may  or  may  not  be  the  best  or  only  method  to  deliver  it.  There  may  be 
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good  reasons  to  use  a  computer  to  replace  a  proctor,  to  deliver  pencil  and  paper  tests,  to 
present  written  text,  to  guide  students  through  a  workbook,  to  provide  extra  practice,  etc. 
Decisions  to  use  a  computer  for  any  instructional  purpose  should  be  based  on  a  careful 
requirements  analysis  coupled  with  good  prescriptive  guidelines  about  what  interventions 
will  achieve  improvement  in  instruction  or  in  its  management.  At  present,  there  is  little 
substantive  evidence  to  guide  the  selection  and  implementation  of  traditional  CBI 
systems,  let  alone  more  sophisticated,  intelligent  ones.  Alternative  means  of  presenting 
instruction  are  possible  and  can  be  as  effective  as  CBI,  and  any  choice  should  be  based  on 
costs  of  developing,  implementing,  and  running  the  different  forms,  balanced  against  their 
expected  outcomes. 


To  construct  instructional  programs,  the  primary  problem  is  the  control  of  the  quality 
of  the  instructional  development  process.  The  use  of  CBI  actually  makes  the  development 
more  difficult.  Attention  must  be  given  during  the  analysis  and  development  phases  to 
instructional  logic  (which  an  instructor  normally  does),  planning  the  student-computer 
interaction  and  interface,  the  types  of  student  response  data  required,  the  schoolhouse 
use,  and  maintainability  of  the  hardware  and  software.  In  addition,  teachers  should  be 
able  to  modify  programs  and  should  learn  to  use  systems  whether  they  develop  instruction 
or  use  canned  programs.  This  description  shows  the  substantial  problems  posed  in 
designing  a  course  using  a  computer  for  training  and  in  programming  the  interactions.  In 
order  to  assist  this  process,  two  developments  are  necessary:  automated  aids  for 
designing  and  developing  training  materials  and  a  coherent  software  distribution  and 
maintenance  system. 

Problems  encountered  in  the  Navy's  instructional  program  development  process 
(Montague  &  Wulfeck,  1982,  1984)  were  largely  due  to  the  variable  quality  of  personnel 
developing  the  course  and  the  lack  of  usable  operating  procedures.  Because  better 
written  guides  cannot  solve  the  problem  completely  (Montague  &  Wulfeck,  1982),  job  aids 
are  needed.  3ob  aids  on  paper  can  help,  but  too  much  depends  on  how  well  the  developers 
learn  and  use  the  methods  and  theory.  If  the  reported  gains  in  productivity  by  using 
automated  job  aids  are  any  guide,  such  aids  for  authoring  instruction  should  make 
substantial  differences  in  the  quality  of  instruction  whether  it  is  given  on-line  or  off-line 
and  in  the  efficiency  of  instructional  program  development. 

^ns*ruc^onal  program  development  aiding  systems  can  assist  with  the 
formidable  record  keeping  problems  involved  in  instructional  development.  More  impres¬ 
sively,  they  can  facilitate  the  development  process  itself  by  guiding  test  and  instructional 
development  and  similar  tasks.  Moreover,  computer-based  systems  can  ensure  that 
guidance  is  followed  by  monitoring  and  evaluating  developers'  performance,  especially  by 
forcing  attention  to  the  delivery  options  available  and  the  trade-offs  among  them  and  by 
assisting  developers  as  they  proceed.  Computer  systems  can  also  provide  training  for 
instructional  developers  who  can  fit  it  into  their  work  schedules.  Finally,  these  systems 
are  essential  for  aiding  implementation  and  utilization  of  CBI.  Most  CBI  users  (schools, 
instructors)  want  the  capability  to  customize  instructional  software.  Aids  for  writing, 
structuring,  and  editing  texts  and  manuals  and  graphics  editors  can  be  incorporated  into 
e  aut  oring  system.  This  provides  an  obvious  mechanism  whereby  developing  science 
can  improve  day-to-day  practice.  New  tools  can  be  included  as  they  are  developed. 

The  transfer  of  CBI  program  software  is  also  a  problem.  There  are  attempts  in 
civilian  education  to  catalog,  annotate,  and  distribute  programs,  but  none  in  the  military. 
Even  with  such  cataloging,  problems  remain.  Many  Navy  activities  buy  computers  that 
are  often  not  compatible  with  each  other.  The  software  they  develop  is  functionally  very 
similar  (e.g.,  programs  to  present  sections  of  text  and  objective  test  questions).  Such 
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programs  require  functionally  the  same  code.  Computer  software  companies  have 
recognized  the  same  sort  of  repetitiveness  in  computer  programs  and  are  developing  ways 
to  use  code  already  developed  in  new  programs.  This  speeds  the  development  process  and 
reduces  errors  by  a  large  factor.  Another  problem  is  that  their  programs  often  are  not 
supported  by  appropriate  authoring  support  and  instructional  management  aids. 

To  address  these  problems,  families  of  CBI  software  should  be  developed  to  support 
CBI  for  a  variety  of  applications.  This  can  be  done  by  developing  libraries  of  CBI 
programs  sufficiently  flexible  to  support  development,  delivery,  and  management  to  meet 
many  instructional  requirements.  The  library  should  also  be  concerned  with 
demonstration  of  and  specifications  for  generic  hardware  systems  capable  of  executing 
library  software,  and  with  planning  for  and  assisting  institutionalization  of  CBI  programs. 
By  providing  transportable,  carefully  tested  CBI  software  and  development  tools, 
compatibility  and  supportability  problems  are  solved,  user  requirements  are  more 
efficiently  addressed,  implementation  and  life-cycle  costs  are  reduced,  standard  data  on 
student  performance  and  CBI  cost-effectiveness  can  be  obtained  for  budget  justification, 
and  acquisition  costs  of  training  can  be  reduced.  Most  importantly,  institutional  software 
libraries  can  achieve  a  "critical  mass"  so  that  evolutionary  improvements  through 
application  of  new  technologies  like  authoring  aids  can  be  achieved. 

The  Department  of  Defense  is  developing  a  library  called  TRIADS  (Dallman, 
Pohlman,  Psotka,  Wisher,  McLachlan,  Wulfeck,  Ahlers,  &  Cronholm,  1983)  to  synthesize 
efforts  in  all  the  services  related  to  CBI  technology.  Initial  programs  in  the  TRIADS 
library  are  those  that  have  already  received  rigorous  test  and  evaluation  within  the 
developing  service.  Later,  programs  will  be  accepted  in  the  library  only  after  analyses 
have  been  conducted  and  only  if  they  interface  with  existing  authoring  and  management 
support  aids  or  include  new  ones.  The  purpose  of  this  effort  is  to  develop  software  and 
instructional  quality  standards  for  programs  to  be  included  in  the  system  library,  to  adapt 
and  enhance  existing  programs  for  this  system  library,  to  demonstrate  the  programs,  and 
to  develop  user  training. 

Efforts  like  TRIADS  provide  straightforward  vehicles  to  put  the  scientific  base  to 
practical  use  and  are  essential  if  CBI  is  to  be  widely  implemented.  However,  current  lack 
of  funds  limits  development. 

In  addition  to  the  development  of  authoring  and  distributing  systems,  programmatic 
efforts  are  needed  in  the  psychological  and  cognitive  sciences  to  develop  the  knowledge 
base  necessary  to  prescribe  instructional  strategies.  For  example,  little  is  known  about 
the  progress  of  and  conditions  that  promote  student  learning.  Support  should  be  given  for 
research  to  investigate  the  design  parameters  and  the  effectiveness  of  various  features 
that  might  be  included  in  CBI  systems. 

A  number  of  questions  need  to  be  studied.  For  example,  most  current  CBI  systems, 
like  education,  use  text  to  communicate  materials.  Graphics  are  sometimes  used,  but  no 
knowledge  base  exists  for  prescribing  when  and  how  to  present  these  materials.  Nor  are 
the  effects  on  student  understanding  known.  Because  the  visual  channel  is  required  in 
many  jobs,  CBI  is  necessary  to  teach  practical  tasks.  However,  current  knowledge  is 
insufficient  to  prescribe  exactly  when  it  is  needed.  A  related  issue  is  the  use  of 
simulation  in  training.  The  psychological  features  of  simulation  fidelity  cannot  be 
specified  to  guide  the  design  of  training  devices. 

The  effects  on  student  learning  and  performance  of  having  training  systems  that 
understand  speech  and  provide  audio  messages  are  unknown.  The  analysis  of  the 
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knowledge  and  information  processing  by  experts  has  been  suggested  to  improve  the 
specifications  of  instructional  objectives.  However,  the  methods  for  doing  such  analyses 
vary  with  researchers.  Considerable  effort  will  be  needed  before  sufficient  information  is 
available  to  guide  task  analysis  used  to  specify  design  requirements  for  CBI.  The  quality 
of  memory  for  the  skills  and  knowledge  learned  using  CBI  has  been  questioned  (Edwards  et 
al.,  1975  cited  in  Kulik  et  al.,  1983).  Most  research  has  not  assessed  retention  as  part  of 
the  evaluation.  Although,  the  few  studies  that  have  tested  retention  show  retention  was 
as  good  for  students  taught  with  CBI  as  for  those  taught  conventionally.  Retention  is 
strongly  dependent  upon  initial  learning  (Hurlock  &  Montague,  1982).  Because  CBI 
produces  better  learning  one  would  expect  better  retention.  These  are  only  a  few  of  the 
issues  that  require  a  substantial  research  effort.  There  is  currently  little  support  for  the 
development  of  this  knowledge  base  necessary  for  the  design  of  effective  training. 


CONCLUSIONS 

CBI  in  its  current  form  can  be  effective  for  certain  Navy  training,  for  example,  at 
remote  sites  where  instructors  are  unavailable,  etc.  It  can  improve  the  effectiveness, 
quality,  and  efficiency  of  training,  and  can  reduce  costs.  CBI,  therefore,  should  be  used  in 
Navy  training,  but  its  introduction  should  not  be  precipitous.  The  CBI  technology  requires 
systematic  development,  and  its  use  requires  careful  analysis  of  the  requirements  for 
training  supported  by  technology,  skillful  creation  of  the  training  materials  by  profes¬ 
sionals,  and  thorough  evaluation  of  the  technology. 


RECOMMENDATIONS 

1.  Develop  usable  procedures  to  determine  when  and  how  to  employ  CBI  effectively 
for  teaching  courses  and  to  manage  its  widespread  distribution. 

2.  Support  the  development  of  automated  systems  to  improve  instructional  program 
development  and  systems  for  collecting  and  distributing  effective  programs. 

3.  Support  programmatic  research  efforts  to  uncover  and  apply  new  knowledge  that 
will  make  CBI  increasingly  effective  for  teaching  complicated  tasks. 
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